Motor vehicle with a driver assistance system and method of operating a driver assistance system

ABSTRACT

A motor vehicle includes a control device and at least one lighting device which is controllable by the control device sequentially in at least two modes which illuminate surroundings of the vehicle differently. At least one camera captures at least two images of the differently illuminated surroundings of the vehicle. The at least two different images are processed by an image processing device for producing a processing result which is transmitted for control of a driver assistance system.

CROSS-REFERENCES TO RELATED APPLICATIONS

This application claims the priority of German Patent Application,Serial No. 10 2012 015 939.0, filed Aug. 10, 2012, pursuant to 35 U.S.C.119(a)-(d), the content of which is incorporated herein by reference inits entirety as if fully set forth herein.

BACKGROUND OF THE INVENTION

The present invention relates to a motor vehicle with a driverassistance system and a method of operating a driver assistance system.

The following discussion of related art is provided to assist the readerin understanding the advantages of the invention, and is not to beconstrued as an admission that this related art is prior art to thisinvention.

The automobile industry increasingly equips motor vehicle with a driverassistance system to assist the driver when exciting standard tasks,like parking or maintaining speed. In addition, systems are more andmore in demand to enable autonomous driving. Driver assistance systemsreceive information from numerous detectors and cameras mounted to thevehicle. Images captured by the cameras are normally processed by animage processing device which generates processing results that are madeavailable to the driver assistance system. A majority ofimage-processing algorithms is based on edge detection. Algorithms foredge detection are based on the recognition of sudden changes in coloror brightness. Therefore, images need to be evenly illuminated whentransmitted to an edge detection algorithm.

When employing such driver assistance systems in poor light conditions,the image area of the camera can be illuminated by lighting devicesmounted on the vehicle. Since illumination from a vehicle is highlydirected, the image may be distorted by shadows or intense contrasts.For example, sensitivity of the camera has to be reduced in the presenceof reflecting or bright or close objects to avoid saturation. This, inturn, causes loss of sensitivity in darken image areas. Moreover,illumination of the vehicle surroundings, using a lighting device on thevehicle, may result in shadows which can be misinterpreted by the imageprocessing device as edge.

It would therefore be desirable and advantageous to address this problemand to obviate other prior art shortcomings.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, a motor vehicleincludes a control device, at least one lighting device controllable bythe control device sequentially in at least two modes which illuminatesurroundings of vehicle differently, at least one camera for capturingat least two images of the differently illuminated surroundings of thevehicle, an image processing device configured for processing the atleast two different images for producing a processing result, and adriver assistance system controlled in response to the processingresult.

The present invention resolves prior art problems by implementing imageprocessing on the basis of several images with different illumination ofthe vehicle surroundings. In this way, it is possible for example toilluminate only parts of the image area of the camera or to illuminatean object from different angles. The captured images include for eachpixel a brightness information. As the images are captured when thevehicle surroundings are differently illuminated, the images storedifferent brightness values for at least portions of the pixels. Thebrightness values of the pixels depend on the intensity of illuminationof an image area, which in turn depends on the direction from which animage area is illuminated. As the image processing device is providedwith several images that have been captured through differentillumination of the vehicle surroundings, it can be recognized forexample whether a sudden brightness transition in an image is the causeof an actual edge or a shadow as the same spot is captured by a secondimage that may not have this shadow for example because the lightingdevice responsible for the shadow is not activated. Thus, solely on thebasis of processing brightness information of several differentlyilluminated images does it become possible to gain further informationabout the surroundings of the motor vehicle.

For example, the afore-mentioned problem of saturation of an image inthe presence of very bright or reflecting objects in the image can beovercome by simply illuminating only parts of the vehicle surroundingsand thus of the image. For example, an image may be initially capturedwith normal lighting and subsequently an image is captured in which thebrightness of illumination is reduced in the area of very bright orreflecting objects. In this case, there are several possibilities forfurther processing. Both images can be combined on a non-linear scale toenable further contrast ranges, on one hand, and an independent edgedetection in both images and a subsequent combination of the informationcan be executed, on the other hand. An independent edge detectionrequires however that new edges may be artificially generated, when animage area is not illuminated or dimly lit. As these edges occur howeverat a known location and with known shape, the presence of such edges canbe easily disregarded by the method.

Capturing several images in different illumination modes also enables adifferentiation between shadows and real edges during edge detection.This can be explained using the simple case of a motor vehicle havingtwo light sources to the left and right on the front side of thevehicle. When directly illuminated by a light cone, an object casts ashadow, i.e. there are regions behind the object with no light. When anobject is illuminated by two parallel light cones at a certain distance,two slightly darkened zones are generated behind the object becausethese zones are illuminated by only one respective light cone. Thisdarkening can easily be recognized as artificial edge during imageprocessing. According to the invention, further images are howevercaptured, with each image being captured by only one of the lightsources of the vehicle being active. Using an active light sourceenables full illumination of one of the two zones previously slightlydarkened, whereas the other one is not illuminated. The edges producedas a result of shadows in the first image partly disappear and partlyare more pronounced in both images which are captured by only one lightsource. Edges that are more pronounced in an image disappear in theother image, and vice versa. As a result, it is now easily possible torecognize edges which are generated by shadows.

When there is no total darkness, there is the possibility to recognizeartificial edges caused by shadows through capturing of some images withreduced brightness of the lighting device. When the contrast of an edgedecreases significantly faster than other contrasts in the image, as aresult of reducing the brightness of the lighting device, it can beassumed that the apparent edge has been generated by a shadow.

The image processing device is thus able in response to a processingresult which is dependent on an edge detection to recognize throughcomparison of images of differently illuminated vehicle surroundingswhether an edge detected at least in one of the images involves anactual edge or an edge caused by a shadow.

According to another advantageous feature of the present invention, thelighting device can include at least one light source capable of beingmodulated. Modulation of the light sources may be implemented in variousways. For example, it is possible to change the beam angle of the lightsource or to change the brightness of the light source. As can be seenwith reference to the afore-described example of a motor vehicle withtwo light sources, it may suffice to simply shut down a single lightsource. Shadows may be recognized and thus false interpretation of edgescan be avoided already in the presence of a certain residual brightnessor a further non-modulated light source.

According to another advantageous feature of the present invention, thebrightness of the light source can be continuously changed.Advantageously, the lighting device can include a plurality of lightsources, with the control device being configured for switching ormodulation of the light sources individually or in groups. Thus, aconfiguration of light sources in groups for example allows detection ofvarious angles of illumination of an object. As described above, the useof two light sources at relatively great distance is oftentimessufficient to clearly distinguish between actual edges and edges causedby shadows. The presence of a greater number of light sources is howeverbeneficial to enable a nearly continuous variation of the angle ofillumination.

For example, it is possible to configure the two front headlights of amotor vehicle with a plurality of light-emitting diodes which illuminatea certain solid angle. This nearly continuous variation of the angle ofillumination is advantageous especially because edges caused by shadows“migrate”. A change of the angle of illumination results in the capturedimage in a rotation of the edge. When capturing the images successivelyat sufficiently short distances, such an edge rotation is normally notto be expected, when actual edges are involved. Thus, edges, caused byshadows can easily be recognized.

According to another advantageous feature of the present invention, thelighting device can include at least one micro-mirror actuator. An arrayof micro-mirrors is hereby illuminated with a light source. Eachmicro-mirror has normally two positions. In one of the positions, lightis deflected onto an absorber. In the other one of the positions, lightis radiated by an optics. When such a system is used instead of a normalheadlight in a motor vehicle, parts of the bundle of rays emitted by theheadlight can be darkened. In this way, illumination by the headlightcan be adapted with very fine resolution.

The presence in a motor vehicle of a plurality of light sources that canbe modulated enables a wide variety of applications in addition to animprovement of contrast and/or edge detection. According to anotheradvantageous feature of the present invention, the processing result canbe a distance interval for an object at at least one pixel, with thecontrol device having at least one further modus to differentlyilluminate the surroundings of the vehicle for operating the lightingdevice, with the image processing device configured to process at leastone further different image, when the distance interval has a widthwhich exceeds a desired value.

Modulation of light sources enables illumination of various solid anglesof the vehicle surroundings. In the presence of a lighting device whichfor example is located distal to a camera and comprised of a pluralityof light sources which illuminate different solid angles, the solidangle between light source and object can be determined at a pixel bymodulating various groups of light sources. A pixel appears the mostbrightest in those images in which all light sources which illuminatethe object at this pixel have maximum intensity. The solid angle betweenlight source and pixel can thus be determined for example byincreasingly darken or deactivating more light sources. A second solidangle is established by image coordinates because each image coordinatecorresponds to a solid angle with respect to the camera.

As the beam angles of the individual light sources are known, distancedetermination requires only determination of which of the light sourcesilluminate the object at the pixel. This is especially possible bycapturing an image at full illumination and subsequently shutting downhalf of the light sources which intensely emit light to the side, i.e.illuminate close objects, and then capturing a further image. In theevent the pixel to be examined in both images is of same brightness, aminimum distance is established. In the event the pixel is markedlydarker in the second image, a maximum distance can be established. In anext shot, this interval can be further subdivided. The change of theillumination and image shot is continued as long as the width of thedistance interval is below a desired value or until another cut-offcondition is reached.

To realize a better distance resolution, the data about the distance canbe interpolated. For simplicity reasons, it shall be assumed thatswitching of two light sources greatly influences the brightness of thepixel. In this case, the image processing device is able to determine,on the basis of the strength of the influence of switching of therespective light source upon the brightness of the pixel, as to whetherthe pixel can be associated to one or the other solid angle. To providea more stable method, it is advantageous to space the light sources atgreat distance. For example, two headlights may be provided which arecomposed of a plurality of light sources.

According to another advantageous feature of the present invention, thedriver assistance system can be configured for autonomous execution ofindividual driving maneuvers, e.g. entering a garage or parking.Assuming for example a possible scenario in which autonomous drivingthrough a gate is desired, whereby the vehicle stands at an unspecifieddistance to the gate. In this case, the situation is captured by thecamera in the vehicle. To enable autonomous driving, it is required torecognize the inner boundaries of the gate, using the camera. When theambient lighting is low, the headlights are activated to illuminate theenvironment. This may produce shadows so that image processingalgorithms, such as e.g. edge detection, may incorrectly interpret thesituation. According to the present invention, shadows can be recognizedthrough change of the illumination and capturing several images. In thisway, it is possible to recognize actual edges to enable an autonomouspassage of the gate. In addition, distances of individual spots can beascertained in accordance with the invention to supplement the pure edgeinformation.

According to another advantageous feature of the present invention, thedriver assistance system may be configured for displaying on a displayan image computed from the processing result of the image processingdevice or complemented by an image of the vehicle surroundings. Forexample, a display of distances of objects is possible. Moreover, e.g.when a rearview camera is involved, the image can be processed so thatedges can be emphasized or 3D models of the vehicle surroundings can becreated.

BRIEF DESCRIPTION OF THE DRAWING

Other features and advantages of the present invention will be morereadily apparent upon reading the following description of currentlypreferred exemplified embodiments of the invention with reference to theaccompanying drawing, in which:

FIG. 1 is a basic illustration of a motor vehicle according to thepresent invention;

FIG. 2 is an exemplary embodiment of the present invention, depicting arecognition of shadows using two lighting devices;

FIG. 3 is a schematic illustration of the present invention, depictingthe presence of light cones using a lighting device with several lightsources;

FIG. 4 is another exemplary embodiment of the present invention,depicting a recognition of shadows using a lighting device with severallighting sources;

FIG. 5 is an exemplary embodiment of the present invention, depicting alighting device with a micro-mirror actuator; and

FIG. 6 is a schematic illustration of the present invention, depicting adistance recognition.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Throughout all the figures, same or corresponding elements may generallybe indicated by same reference numerals. These depicted embodiments areto be understood as illustrative of the invention and not as limiting inany way. It should also be understood that the figures are notnecessarily to scale and that the embodiments are sometimes illustratedby graphic symbols, phantom lines, diagrammatic representations andfragmentary views. In certain instances, details which are not necessaryfor an understanding of the present invention or which render otherdetails difficult to perceive may have been omitted.

Turning now to the drawing, and in particular to FIG. 1, there is showna basic illustration of a motor vehicle according to the presentinvention, generally designated by reference numeral 1. Mounted to thefront region of the motor vehicle 1 is a lighting device 2 and a camera4. Images captured by the camera 4 are sent to an image processingdevice 5. Processing results generated by the image processing device 5are transmitted to a driver assistance system. The lighting device 2includes a plurality of light sources 7, e.g. LEDs. The individual lightsources 7 are operated by a control device 3. In order to capture imagesof the vehicle surroundings in at least two illuminations, the lightsources 7 of the lighting device 2 are operated by the control device 3immediately sequentially in at least two different modes whichdifferently illuminate the vehicle surroundings. In addition, thecontrol device 3 sends also a signal to the image processing device 5 toprovide information about the illumination of the images. The imageprocessing device 5 is then able on the basis of processing a predefinednumber of images to send a processing result to the driver assistancesystem 6. The driver assistance system 6 may use this processing resultfor autonomous piloting of the vehicle for example.

It is also possible for the image processing device 5 to instruct thecontrol device 3 to produce further illumination situations in responseto the result of the image processing so that further images can beprocessed. This is especially useful for example, when the distance ofan object should be determined or there is still ambiguity about theauthenticity of a detected edge.

FIG. 2 shows an exemplary embodiment for the recognition of shadows whentwo lighting devices are involved. Mounted to a front region of motorvehicle 1 is a camera 4 in midsection, a lighting device 2 to the leftof the camera 4 and a lighting device 9 to the right of the camera 4. Anobstacle 8 is shown at a distance to the front of the motor vehicle 2.

When both lighting devices 2, 9 are activated, zones are generated tothe left and right of the obstacle 8 which are illuminated by only onelighting device, respectively, and involve shadow 10 produced by thelighting device 2 and shadow 11 produced by the lighting device 9. Arapid change in brightness occurs at the edges of these zones and in alllikelihood is detected as edge. This needs to be avoided and is achievedby capturing in addition to the first image a further image whichinvolves activation of only the lighting device 2 and still anotherimage which involves activation of only the lighting device 9. As aresult, the first of these images fully shows the shadow 10 of thelighting device 2 and generates a very contrast-rich edge, while theshadow 11 of the second lighting device 9 is not visible as the lightingdevice 9 is switched off. The other image shows precisely the reversesituation. Thus, edges caused by shadows are thus clearly more visiblewith high contrast in one image than before and not visible at all inthe other image. As a consequence, such edges can thus be unambiguouslyrecognized as the result of a shadow and thus can be disregarded forimage processing.

FIG. 3 shows a schematic illustration of a cursorily shown motor vehicle1, depicting the presence of light cones using a lighting device 2 withseveral light sources which are arranged in offset relationship andilluminate a slightly rotated solid angle range. By way of example, theedge of the light cone 13 of the light source 12 is separately marked.As can be clearly seen, the light sources in their entirety illuminate abroad solid angle of the vehicle surroundings. This solid angle can belimited by deactivating individual light sources.

FIG. 4 shows another exemplary embodiment of the present invention,depicting a recognition of shadows using a lighting device with severallighting sources. In this example, the obstacle 8 is exclusivelyilluminated by a lighting device 2. By way of example, the edges of theshadow 15 of the outer light source 14 and the shadow 17 of the innerlight source 16 are shown. When activating the light sourcessuccessively from outwards inwardly, the edges of the shadow migratebetween these positions. This results in a virtually continuous shift ofthe edge with a change of the light sources. Therefore, these edges canbe easily recognized as caused by shadows and can be disregarded whendetermining the processing result of the image processing.

FIG. 5 shows an exemplary embodiment of the present invention, depictinga lighting device with a micro-mirror actuator. A light source 18 whichis integrated in a reflector 19 hereby illuminates a micro-mirroractuator 20. The micro-mirror actuator 20 includes an array ofmicro-mirrors which can be switched back and forth between two states. Amicro-mirror in initial or OFF state 21 reflects light in the directionof an absorber 23. A micro-mirror in ON state 22 reflects light in thedirection of an optics 24 which projects the light cone. When allmicro-mirrors assume the ON state, a light beam with maximum solid angleillumination is generated. By switching individual micro-mirrors orgroups of micro-mirrors to the ON state, the light beam can be tailoredsuch that portions of the light beam can be obscured. As a result, aselective illumination is realized.

FIG. 6 is a schematic illustration of the present invention, depicting adistance recognition. For sake of simplicity, all light sources 7 of thelighting device 2 are arranged on a straight line. Continuous lines 25indicate the center of the light cone of a light source, whereas brokenlines 26 mark the transition zone between light cones of neighboringlight sources. The dash-dot line 27 shows the solid angle which isassociated to the camera pixel. For sake of clarity, a central pixel isselected to correspond with a straight line extending in traveldirection.

When an obstacle 8 can be recognized on the camera pixel, the distanceof the obstacle 8 can be determined. For that purpose, successive lightsources of the left and right lighting devices can be deactivated,respectively, for example from outwards inwardly. When determining asignificant decrease in the brightness of the pixel at any point intime, it is assumed that the object is located in the solid angle of thevehicle surroundings which this light source illuminates. It can thus beascertained in the shown example that the object is in the area of thelight sources 28, 29, 30, 31. As activation of any of these lightsources causes a same change in brightness of the pixel, inference canbe made that the object is located in the edge zone of the neighboringlight cone of the light sources 28, 29 and 30, 31. As a result, twosolid angles in relation to the lighting sources and a solid angle inrelation to the camera are known for the positions of the object. Thisallows a redundancy in determination of the object position. As a resultof the redundancy, errors in the distance measurement can easily bedetected.

While the invention has been illustrated and described in connectionwith currently preferred embodiments shown and described in detail, itis not intended to be limited to the details shown since variousmodifications and structural changes may be made without departing inany way from the spirit and scope of the present invention. Theembodiments were chosen and described in order to explain the principlesof the invention and practical application to thereby enable a personskilled in the art to best utilize the invention and various embodimentswith various modifications as are suited to the particular usecontemplated.

What is claimed as new and desired to be protected by Letters Patent isset forth in the appended claims and includes equivalents of theelements recited therein:
 1. A motor vehicle, comprising: a controldevice; two lighting devices located at opposite sides of the motorvehicle and controllable by the control device sequentially in at leasttwo modes which respectively illuminate surroundings of the vehicledifferently so that first one of the lighting devices at one of theopposite sides of the motor vehicle is activated and thereafter anotherof the lighting devices at another of the opposite sides of the motorvehicle is activated to produce at least two images of the differentlyilluminated surroundings of the vehicle, wherein each of the lightingdevices has several light sources which are arranged in an offsetrelationship and illuminate a rotated solid angle range; at least onecamera for capturing the at least two images of the differentlyilluminated surroundings of the vehicle; an image processing deviceconfigured for processing the at least two different images forproducing a processing result, wherein the processing result is adistance interval for an object at at least one pixel, said controldevice having at least one further modus to differently illuminate thesurroundings of the vehicle for operating at least one of the lightingdevices, said image processing device configured to process at least onefurther different image, when the distance interval has a width whichexceeds a desired value; and a driver assistance system controlled inresponse to the processing result, wherein the driver assistance systemis configured for autonomous execution of entering a garage or parking.2. The motor vehicle of claim 1, wherein the lighting devices include atleast one light source capable of being modulated.
 3. The motor vehicleof claim 1, wherein the lighting devices include a plurality of lightsources, said control device being configured for switching ormodulation of the light sources individually or in groups.
 4. The motorvehicle of claim 1, wherein the lighting devices include at least onemicro-mirror actuator.
 5. The motor vehicle of claim 1, wherein thedriver assistance system is configured for autonomous execution ofindividual driving maneuvers.
 6. The motor vehicle of claim 1, whereinthe driver assistance system is configured for displaying on a displayan image computed from the processing result of the image processingdevice or complemented by an image of the vehicle surroundings.
 7. Amethod of operating a driver assistance system of a motor vehicle,comprising: capturing a first image by a camera of a first area of avehicle surroundings illuminated by a first lighting device at one oftwo opposite sides of the motor vehicle when operating in a first mode;capturing a second image by the camera of a second area of the vehiclesurroundings illuminated by a second lighting device at another of thetwo opposite sides of the motor vehicle when operating in a second mode,wherein each of the lighting devices has several light sources which arearranged in an offset relationship and illuminate a rotated solid anglerange; processing the first and second images by an image processingdevice to produce a processing result, wherein the processing stepincludes determining a distance interval for an object at at least onepixel, and capturing a third image by the camera of a third area of thevehicle surroundings illuminated by at least one of the lighting deviceswhen operating in a third mode and processing the third image by theimage processing device, when a width of the distance interval exceeds adesired value; transmitting the processing result to the driverassistance system; and autonomously executing a driving maneuver by thedriver assistance system in response to the transmitted processingresult, wherein the driving maneuver involves entering a garage orparking.
 8. The method of claim 7, wherein the driver assistance systemoperates in either of two ways, a first way in which the driverassistance system computes an image of the vehicle surroundings inresponse to the transmitted processing result and subsequently displaysthe computed image on a display, a second way in which the driverassistance system supplements the processing result with at least one ofthe first and second images and subsequently displays the supplementedimage on a display.
 9. The motor vehicle of claim 1, wherein all of thelight sources of each of the lighting devices illuminate the solid anglerange which is a broad angle range of the vehicle surroundings, and someof the light sources of each of the lighting devices are deactivatableto limit the angle range.
 10. The method of claim 7, wherein all of thelight sources of each of the lighting devices illuminate the solid anglerange which is a broad angle range of the vehicle surroundings, and someof the light sources of each of the lighting devices are deactivatableto limit the angle range.